This invention relates to video special effects, and more particularly to an improved apparatus and method for the calculation of image compression to optimize the amount of anti-aliasing filtering that is performed over different parts of a video image that is to be mapped from 2-D to 3-D and back to 2-D.
A three-dimensional digital video effects system maps an incoming video image that is two-dimensional into a three-dimensional space and then performs a perspective projection of the mapped image back into two dimensions for eventual display. The incoming (source) image is in the form of a two-dimensional array of picture elements, or pixels. This array of pixels is resampled into an output (target) image, which is also a two-dimensional array of pixels.
Because the mapping (transformation) of the source image into three-dimensional space and back into the target image results in varying amounts of compression (resizing) of the source image as it appears in the target image, the resampling operation often results in an undersampling of the source image detail as it appears in the target image. Image undersampling can produce the effect commonly known as "aliasing". Aliasing can arise whenever the two-dimensional frequency content of the source image is greater than half of the two-dimensional sampling rate of that image by the resampling process in forming the target image. The visual effect that results from aliasing is a displeasing graininess in the resulting target image.
Aliasing can be reduced by low-pass filtering the source image prior to resampling, so as to reduce its high frequency content at and above the resampling rate. Excessively low-pass filtering the incoming video image, with a cutoff frequency that is much below the resampling frequency, causes a blurring of the resulting target image. It is therefore important to know exactly how much compression of the image has taken place, so that the anti-aliasing low-pass filtering can be minimized and yet be effective. A useful rule is that the cutoff frequency of the anti-aliasing filter, normalized to the sample rate of the incoming image, be 1/(2.0.times.the amount of compression).
Sending an image to a farther perceived depth via an effects transform image makes the target image appear smaller than the source image. An effects system can also make the target image look smaller by resizing the source image through remapping. Without any additional depth cues it is impossible to tell if a compressed target image is a result of resizing or of depth displacement. For the remainder of this document we will refer to compression from resizing and compression from depth displacement as depth-based compression.
The amount of compression of the source image as it appears in the target image arises from both the perception of depth and of perspective skew, each of which arises from the apparent perspective in which the source image appears in the target image. Referring to FIG. 1A, it can be seen that the "A" on the left appears smaller and therefore compressed relative to the one on the right. And, referring to FIG. 1B, it can be seen that the "B" at the top left and the "B" at the top right have undergone some perspective or vanishing point compression relative to the "B" at the top middle. All three "B"s in the top of this image have undergone the type of compression seen in FIG. 1A, but the ones on the sides are also affected by additional compression due to vanishing point (perspective) skew that makes the rectangular source image appear as a trapezoid.
Because both of these factors, perception of depth and perspective skew, vary from point to point within a transformed image, determining the cut-off frequency for the low-pass filtering process is preferably accomplished locally within the image, ideally on a pixel-by-pixel basis, so that only the minimum necessary amount of filtering is performed. Thus, the compression factors should be calculated in video real time, and it is naturally desirable to be able to calculate these compression factors with a minimum of computational resources.
Performing the function of finding local compression factors to aid in optimizing anti-aliasing filtering is not new. For example, the DPM-1 (also known as "Kaleidoscope") video effects system made by The Grass Valley Group, Inc. of Grass Valley, Calif., provides a system for finding local compression factors to aid in controlling anti-aliasing filtering; however, in that system a great deal of computational resources are employed to attain the desired filter compression factor control signals.